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1 Introduction 



(N . . ^ _ 

\ Zt and 7t production with Z — > I'^l and t — > Wb Ivb provides the best de- 

' termination of top flavour-changing neutral couphngs at the LHC. The bounds on 

. tc couphngs eventuahy derived from these processes are similar to those expected 

0^ \ from top decays, while the limits on tu couplings are better by a factor of two. The 

■ other significant Z and W decay modes are also investigated. 

■ PACS: 12. 15. Mm, 12.60.-i, 14.65.Ha, 14.70.-e 

ix. 

> 

X 

d • Future colliders will explore higher energies looking for new physics. Even if there is not 

a new production threshold the top quark will probe the physics beyond the Standard 
Model (SM). Large e^e~ and hadron colliders, in particular the CERN Large Hadron 
Collider (LHC), will be top factories allowing to measure the top couplings with high 
precision. Both types of machines are complementary for if the LHC will produce many 
more tops, e~^e~ linear colliders will face smaller backgrounds. Nevertheless the most 
realistic estimates indicate that the precision which can be reached for flavour- changing 
neutral (FCN) couplings can be up to a factor of three better at the LHC with a luminosity 
of 100 fb-i [1,2] than at a 500 GeV e+e" linear collider with a luminosity of 100 fb'^ (LC) 
[3]. The top couplings which can signal to new physics can be diagonal [4] or off-diagonal. 
We will concentrate on the latter. FCN couplings are very suppressed within the SM but 
can be large in simple extensions [5]. Hence they are a good place to look for departures 
from the SM. However present data only allow for FCN couplings large enough to be 
observed at future colliders for the top quark. In this paper we investigate the sensitivity 
of the LHC to top FCN gauge couplings. This has already been studied for top decays 
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[1,2]. We look at Vt production, V = Z.j, induced by anomalous top couplings [6,7], 
which, as we shall show, provide better limits. 

In order to describe FCN couphngs between the top, a light quark q — u,c and a Z boson, 
a photon A or a gluon G" we use the Lagrangian [8] 



+ef(A« - iAj?75)^^?A'^ + gstiCS' - iCSh^)'-^T'^QG'^' + h.c. , (1) 
^ ^ rrit ^ ^ rrit 

where Pr,l = (1 ±75)/2 and are the Gell-Mann matrices satisfying Tr (TT^) = 5"72- 
The couplings are constants corresponding to the first terms in the expansion in momenta. 
This effective Lagrangian contains 7^ terms of dimension 4 and cr^j, terms of dimension 5. 
The a^i, terms are the only ones allowed by the unbroken gauge symmetry, SU(3)^xU(l)g. 
Due to their extra momentum factor they grow with the energy and make large colliders 
the best place to measure them. They are absent at tree-level in renormalizable theories 
like the SM, where they are also suppressed by the GIM mechanism [9]. However the 
effective couplings involving the top quark can be large in models with new physics near 
the electroweak scale. In effective theories with only the SM light degrees of freedom 
the 7^ terms above also result from dimension six operators after electroweak symmetry 
breaking. However, if the scales involved are similar, as happens with the top quark mass 
and the electroweak scale, the 7^ terms can be also large [10]. This is the case in simple SM 
extensions with vector-like quarks near the electroweak scale [11]. Although rare processes 
strongly constrain FCN couplings between light quarks [12], the top can have relatively 
large couplings to the quarks u or c, but not to both simultaneously [5]. Thus the third 
family seems the best place to look for SM departures and Eq. (1) is the lowest order 
contribution to trilinear top FCN gauge couplings of any possible extension. It is then 
important to measure them at the LHC [13]. 

The vertices in Eq. (1) are constrained by the nonobservation of the top decays t ^ qV 
[14] and t ^ qg [15] at Tevatron, implying the present direct limits 



Xtq=\ 






< 0.84, 


Ktq=\ 




|2 1 |k-(^)|2 


< 0.78 , 


Xtq=] 






< 0.33, 


Ctq=\ 


/icS'i 


2 _|_ |/"(^)|2 


< 0.15 



(2) 

at 95% C. L. (Unless otherwise stated, all bounds in this paper have this confidence 
level.) The sensitivity of the LHC and LC for some of these couplings has been studied 
for different processes in a series of papers. At the LHC with an integrated luminosity of 
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100 the expected limits from top decays are Xtq < 0.017 [1], Xtq < 0.0035 [2]. The 
LC will probe the electroweak couplings in the process e~^e~ — tc, obtaining eventually 
Xtq ^ 0.051, Ktq < 0.015, Xtq < 0.011 [3]. The best limits on the gtq vertices are expected 
from single top production at LHC [16], Qu < 0.0004, < 0.0009. In the following we 
show that the process gq Vt gives competitive bounds on the Vtc couplings, Xtc < 
0.023, Ktc < 0.016, Xtc < 0.0065, and the best limits on the Vtu vertices, Xtu < 0.011, 
Ktu < 0.0063, Xtu < 0.0021. This process can take place through the s- and t-channel 
diagrams in Fig. 1 via Vtq couplings, or via gtq couplings through the diagrams in Fig. 2. 
However, in the second case the best limits on the strong anomalous couplings (tu < 
0.0018, (tc < 0.0037 are less stringent than those derived from tj production. 




Fig. 1. Feynman diagrams for gq — > Vt via Vtq FCN couplings with V = Z,^. The Z boson and 
the top quark are off-shell and have the SM decays. 
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q V q V 

Fig. 2. Feynman diagrams for gq Vt via gtq FCN couplings with V = Z,j. The Z boson and 
the top quark are off-shell and have the SM decays. 

It must be noticed that all upper bounds we shall derive will be obtained following the 
Feldman-Cousins construction [17] to obtain 95% C. L. intervals, assuming that the num- 
ber of observed events n equals the expected background rib- For the numerical evaluation 
of the confidence intervals we use the PCI package [18]. The intervals obtained with the 
Feldman-Cousins construction are similar to standard 95% C. L. upper limits on a Poisson 
variable with known background [19] but they give less restrictive bounds (see Table 1). 
Another possibility is to use the 95% C. L. (1.96 a) discovery significance Ug/ y/ub > 1.96, 
with Us the expected number of signal events (see for instance Ref. [20]). The limits ob- 
tained are similar to standard 95% upper limits when the Poisson distribution can be 
approximated by a Gaussian {rib > 5). The quoted limits in Refs. [1-3,15,16] have been 
obtained using the statistical significance ng/y/Us + > 3. This is a conservative esti- 
mate with a 99% C. L. and weakens the bounds on the anomalous couplings given here 
by a factor ~ a/2 (see Table 1). This is part of the improvement found. 
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Ub Us (Ref. [17]) Us (Ref. [19]) (Ref. [20]) (Refs. [1-3,15,16]) 







3.09 



3.00 







3.84 



5 



6.26 



5.51 



4.38 



6.71 



10 



7.82 



6.97 



6.20 



8.41 



15 



9.31 



8.09 



7.59 



9.75 



Table 1 

95% C. L. upper bounds on signal events Ug for samples of Ub backgound events obtained using 
four different prescriptions. 

The sensitivity to top FCN couplings varies with the Z and W decay modes. These are 
collected in Table 2, together with their branching fractions and main backgrounds. We 
neglect nonstandard top decays since they are a small fraction to start with. Moreover, this 
approximation becomes better when smaller arc the upper limits on anomalous decays, 
which is the practical case we are interested in. In the leptonic modes we only consider 
decays to electrons and muons, but a good r tagging efficiency increases the leptonic 
branching ratios in Table 2 and improves their statistical significance. The relevance of 
the different decay channels results from the balance between the size of the signals, the 
corresponding backgrounds and their statistical significance, and varies substantially from 
the Fermilab Tevatron to the LHC For instance, at Tevatron Run II with a luminosity 
of 2 fb~^ the vPjjb decay mode is the most interesting one due to its branching ratio, 
with 235 signal and 22 background events after kinematical cuts for Xtu = 0.84 [6]. 
At this collider, the l^l^lvh channel has a much larger signal to background ratio (28 
signal and 0.07 background events before kinematical cuts for = 0.84) but still its 
statistical significance is lower and the bound obtained from the former decay mode is 
more restrictive. However, this behaviour is reversed at the LHC, where the increase in 
energy and luminosity makes the l^l~li'h channel the most sensitive one. 

Other decay modes also provide stringent limits. The l^l~jjb channel gives competitive 
but worse results than l'^l~lub at the LHC. The modes with hadronic Z decay, in partic- 
ular jjlvb, which gives significant bounds at Tevatron, have huge backgrounds at higher 
energies and luminosities: the Wt background becomes more important due to the larger 
b content of the proton at LHC energies, and tt, with t{t) jjb, i{t) Ivb and one jet 
undetected, also grows rapidly with the energy. Thus the jjlfb channel is not interesting 
any more. The situation improves considering only Z — > 66 and requiring three b tags, 
but the results are worse than for the channels with Z I'^l" . The vVjjb mode also 
has as backgrounds Wt and fi but we will discuss this signal for illustration, since this 
mode is the best one at Tevatron. The remaining decay channels uulub, jjjjb and bbjjb 
have too large backgrounds and will not be treated here. The 7/Z/6 decay mode is the 
most interesting one in jt production, due to its small jWj background, ^jjb gives also 
significant but less restrictive limits. 

It is worth to note that b tagging plays an essential role in enhancing single top signals. 
The approximately conserved 6-Parity in the SM [22] allows in practice to get rid of large 
backgrounds with an even number of b quarks in the final state. 



4 







W 


^ li' 




W^ 


33 




Final state 


Backgrounds 


Final state 


Backgrounds 


z i+i- 
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Z^hh 


hhlvh 


3.3% 
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10.2% 


bbjjj 


^ 33 


jjlub 


15.2% 


tt, Wt, Wjjj 


jjjjb 


47.4% 


33333 


7 (stable) 


'ylub 


21.8% 


jWj 




67.8% 


1333, iWj 



Table 2 

Branching ratios and main backgrounds for the different Z and W decay channels, j and jj 
include b and bb when possible (we neglect the small Cabibbo-Kobayashi-Maskawa mixing angles 
K&,K6~0 [21]). 

This paper is organized as follows. In Section 2 we discuss how the different signals 
and backgrounds are generated. In Sections 3 and 4 we analyze Zt and signals. We 
summarize in Section 5. 



2 Signal and background simulation 

In general Zt production yields five fermion final states with at least one h quark ffffb. 
We evaluate these signals with the exact matrix element for the s- and t-channcl diagrams 
gq ^ Zt ^ ZWb — > ffffb (see Figs. 1 and 2). The SM diagrams gq ZWb are much 
smaller in the phase space region of interest and suppressed by small mixing angles, so we 
neglect them in the signal evaluation. We also ignore interferences from identical fermion 
interchange in the final state. All fermions are assumed massless except the top quark, and 
we assume only one type of coupling to be nonzero at a time. We evaluate '-ft production 
in a similar way, with the s- and t-channel diagrams gq ^ jt ^ jffb. t and i production 
are summed up in all cases. 

The VWj backgrounds are evaluated considering gq^ — ^ VWqd, gqd VWqu and quqd 
VWg (with qu = u,c and qd = d, s, b) plus the charge conjugate processes. We calculate 
the matrix elements for gq^ VWqd, including the eight SM diagrams and decaying the 
Z and W afterwards. The matrix elements for the two other processes are obtained by 
crossing symmetry. Our results are consistent with those given in Ref . [23] . 

In order to calculate the Zjjj, Zbbj, Wjjj and Wbbj backgrounds we use VECBOS [24] 
modified to include the energy smearing and trigger and kinematical cuts. We also include 
routines to generate the kinematical distributions. To generate the jjjj background we 
have further modified VECBOS to produce photons instead of Z bosons. This is done by 
introducing a 'photon' with a small mass uij = 0.1 GeV and substituting the Z couplings 
by the photon couplings everywhere. The total width of such 'photon' is calculated to be 
= 1.73 • 10~^ GeV, with an e+e~ branching ratio equal to 0.15. We have checked that 
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the results are the same for a heavier 'photon' with — 1 GeV and = 1.73 • 10 ^ 
GeV. 

We have also to evaluate Wt production, which is similar to the signal Zt, but proceeds 
through the SM process gh — > Wt — > WWb — > ffffb, and is calculated analogously. 

tt production with some particles missed by the detector must be also considered [25]. 
Finally, Wj, jjjjj and bbjjj production is so large that makes unnecessary a detailed 
discussion of the corresponding signals and of the backgrounds themselves. Other possible 
small backgrounds are neglected [7,20] . 

Wc include throughout the paper a K factor equal to 1.1 for all processes [26] except for 
tt production for which we assume K = 2.0 [27]. We use MRST structure functions set 
A [28] with = s. The cross sections have some dependence on the structure functions 
chosen but not the trend of our results. 



After generating signals and backgrounds we imitate the experimental conditions with a 
Gaussian smearing of the lepton (/), photon (7) and jet (j) energies [29], 

A£;''T 10% AE^ 50% 

® 0.3% , -— = ^= ® 3% , (3) 



E^'^ y/E^ ■ ' Ei y/E^ 



where the energies are in GeV and the two terms are added in quadrature. For simplicity 
wc assume that the energy smearing for muons is the same as for electrons. Wc then apply 
detector cuts on transverse momenta pr, pseudorapidities r] and distances in (7;, 4>) space 
AR: 

p^T > 15 GeV , > 20 GeV , pi > AO GeV 

IV'^-'^I < 2.5 , ARjj,ij,^i,^j>OA. (4) 



For the Wt and tt backgrounds, we estimate in how many events we miss the charged 
lepton and the b or any other jet demanding that their momenta and pseudorapidities 
satisfy < 15 GeV, < 20 GeV or \ri^'^\ > 3. 

For the events to be triggered, we require both the signal and background to fulfil at least 
one of the trigger conditions [30]. For the first LHC Run with a luminosity of 10 fb~^ (L), 

• one jet with px > 180 GeV, 

• three jets with pt > 75 GeV, 

• one charged lepton with pt > 20 GeV, 

• two charged leptons with pt > 15 GeV, 

• one photon with pt > 40 GeV, 

• missing energy 50 GeV and one jet with px > 50 GeV, 
and for the second Run with 100 fb~^ (H), 
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• one jet with pt > 290 GcV, 

• three jets with Pt > 130 GeV, 

• one charged lepton with pt > 30 GeV, 

• two charged leptons with px > 20 GeV, 

• one photon with pt > 60 GeV, 

• missing energy ^t> 100 GeV and one jet with pt > 100 GeV. 

Finally, we require a tagged b jet in the final state taking advantage of a good b tagging 
efficiency ~ 60% perhaps better than the one finally achieved [31]. There is also a small 
probabihty ~ 1% that a jet which does not result from the fragmentation of a 6 quark is 
misidentified as a 6 jet [32]. b tagging is then implemented in the Monte Carlo routines 
taking into account all possibilities of b (mis)identification. As we shall see, this reduces 
substantially the backgrounds. 

To conclude this Section let us emphasize the importance of performing the full 2 — > n 
body calculation with the intermediate particles off-shell. To illustrate the relative im- 
portance of considering the intermediate particles off-shell and of simulating the detector 
conditions with a Gaussian smearing of jet, charged lepton and photon energies we con- 
sider the reconstructed Z boson mass Mp^ for its leptonic decay, with M^^^ defined as the 
two lepton invariant mass (for instance in the Zt decay mode l~^l~jjb). In Fig. 3 we plot 
the corresponding distributions for Z off- and on-shcU, including in both cases the energy 
smearing. We observe that at LHC, for these detector resolutions and lepton energies, 
the effect of the off-shellness is more important than the energy smearing. (Of course the 
same applies to the intermediate t and W.) Applying kinematical cuts on reconstructed 
masses (or any related variable such as the sum of prs of the decay products) without 
allowing the corresponding particles to be off-shell at least in the signal would lead to 
very optimistic limits. In the case of the Z hadronic decay, the only way to distinguish 
Zt production from the copious Wt production is requiring a Z reconstructed mass not 
consistent with the W mass. Hence it is essential to generate this signal and background 
with Z and W off-shell. 



3 Zt production 

Although anomalous Zt production is a tree level process with a strong vertex (see Fig. 1) 
we will be eventually interested in small anomalous Ztq couplings which give small cross 
sections. It is then important to perform a detailed analysis to look for the statistically 
most significant decay channels. As we will show, these are the modes with Z decaying 
leptonically, l^l~lub and l'^l~jjb. We discuss the two channels in turn. Of the modes with 
Z decaying hadronically, the jjli^b decay mode (which has a statistical significance similar 
to l^l~lvb at Tevatron Run 11) has in practice too large Wt and ti backgrounds at the 
LHC to be interesting. However, considering only Z ^ bb and then requiring three b tags 
these backgrounds are reduced and the bblub signal becomes the most relevant one with 
Z decaying into hadrons. Finally we discuss the lypjjb signal, which is the best one at 
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Tevatron Runs I and II but at LHC has too large Wt and tt backgrounds. For each decay 
channel limits on anomalous gtq couplings can be derived through the process in Fig. 2. 
We quote without discussion the best limits from Zt production, which are also provided 
by the three charged lepton decay l'^l~li'b. 

3.1 l^l'lvh signal 

The l^l~lvh mode is the best decay channel to search for anomalous Ztq couplings at 
LHC. Its branching ratio is very small, 1.5%, but its only background is ZWj production, 
with j misidentified as a 6 with a probability of 0.01. The true h production from initial 
u and c quarks is suppressed by the Cabibbo-Kobayashi-Maskawa matrix elements |Vubp 
and \ Vch\^ [21], respectively, and is negligible. For better comparison here and throughout 
this paper we normalize the signal to Xtq — 0.02 and Ktq — 0.02, and to be definite we 
fix the ratio X^^/ = 4/3. To perform kinematical cuts on the signal and background 
we must first identify the pair of oppositely charged leptons l^l^ resulting from the Z 
decay. There are two such pairs and we take that with invariant mass M^^^ closest to the 
Z mass. We do not perform any kinematical cut on because obviously signal and 
background peak around Mz and use this procedure only to identify the charged lepton I 
resulting from the W decay. We then make the hypothesis that all missing energy comes 
from a single neutrino with p*^ = {E^ ."pT^Pt)-, and the missing transverse momentum. 
Using {p^ + p^Y = fi'^*^ ^^'^ solutions for p*^, and we choose the one making 

the reconstructed top mass m\^^ = ^ (p^ + p'^ + p'^y closest to mj. In Fig. 4 we plot this 
distribution for the gu — > Zt signal and background. We observe that the background has 
a maximum near m^. This is because in this rrit reconstruction method we first impose 
(p' + p")"^ = and then we choose the best of the two possible ml'^'^ values. Other 
interesting kinematical variables are the total transverse energy Ht in Fig. 5, defined in 
general as the scalar sum of the pts of all jets, photons and charged leptons plus ^t, and 
, the reconstructed transverse momentum of the Z boson, plotted in Fig. 6. 

To enhance the signal to background ratio we apply the kinematical cuts 
and p^ in Table 3. In addition we require I^t> 5 GeV to ensure a meaningful top mass 
reconstruction. The higher luminosity of Run H allows more stringent cuts that eliminate 
90% of the background while retaining more than 60% of the signal. The total number of 
signal and background events for Runs L and H with integrated luminosities of 10 fb~^ and 
100 fb~^, respectively, is collected in Table 4, using for the signal Xtq = 0.02, Ktq = 0.02. 
Note that for Run L the trigger is redundant since all events passing the detector cut 
pI > 15 GeV automatically satisfy the leptonic trigger. Comparing the numbers before 
kinematical cuts it can be also observed that in Run H the trigger has little effect, due 
to the presence of three charged leptons in the final state. To derive upper bounds on the 
coupling constants we use the prescriptions of Ref. [17] (similar to those applied in Ref. 
[14] to obtain the present Tevatron limits). The contributions from u and c quarks must 
be summed up if a positive signal is observed. However, if there is no evidence for this 
process, independent bounds for each quark and coupling can be obtained, Xtu < 0.022, 
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Xtc < 0.045, Ktu < 0.014, Ktc < 0.034 after Run L and Xtu < 0.011, < 0.023, 
f^tu < 0.0063, Ktc ^ 0.016 after Run H. (The expected limit from top decay after Run H 
is Xtq < 0.017.) 

Variable Run L Run H 
mp^ 150-200 160-190 
Ht > 200 > 260 

Pj, > 50 

Table 3 

Kinematical cuts for the decay channel. The masses, energies and momentum are in 

GeV. 

Run L Run H 

before after before after 







cuts 


cuts 


cuts 


cuts 


gu 




5.0 


4.8 


49.4 


31.6 


gc 




1.1 


1.1 


11.4 


6.5 


gu 


zt (cr^i,) 


11.1 


10.9 


111 


88.1 




-^Ztia^,) 


2.0 


1.9 


19.5 


14.4 




ZWqu 


4.9 


1.4 


49.2 


5.0 




ZWqa 


5.5 


1.4 


54.8 


5.1 




ZWg 


4.7 


1.1 


47.4 


4.0 



Table 4 

Number of l~^l~lub events before and after the kinematical cuts in Table 3 for the Zt signal and 
backgrounds. We use Xtq = 0.02 and = 0.02. 

One may wonder whether it would be useful to exploit the characteristic q'^ behaviour of 
the couplings requiring large transverse momenta to obtain more stringent bounds. 
In this case, it makes little difference and requiring Ht > 360 GeV in Run H only reduces 
the Ktu hmit to 0.006. 

This decay channel can be also used to constrain the gtq couplings through the process 
in Fig. 2. Proceeding in the same way as before we obtain < 0.0069(0.0030), Qc < 
0.017(0.0078) after Run L (H). 

3.2 l^l'jjb signal 

This is the most interesting channel with hadronic W decay. At Tevatron this mode is 
surpassed by the vvjjh channel due to the relatively low statistics available and its greater 
branching ratio, but this is not the case at LHC. The main background for l^l^jjb is Zjjj 
production with a jet misidentified as a 6. The second background is Zbbj production with 



9 



only one b tagged. The ZWj background in this case is much smaller but we take it into 
account at the end for comparison. 

To reconstruct the signal we first perform b tagging with the corresponding probabilities 
of 0.6 for b jets and 0.01 for non b jets, and require only one b tag. This reduces the signal 
by a factor of 0.6, the largest background Zjjj by 0.029 and the Zbbj background by 0.48. 
After tagging the b, assumed to come from the top quark decay, the two remaining jets 
are assigned to the W, and the W reconstructed mass is defined by their invariant 
mass. In this case the top reconstructed mass m^^^ is simply the invariant mass of the 
three jets. These two invariant masses are not independent, {ml^Y — (M^^)^ + 2p^ • p**, 
and the kinematical cuts are less effective than for the previous signal. For l^l~jjb we also 
perform cuts on and on the transverse momenta of the fastest jet pj"^^^, the fastest 
lepton p''j^^^ and the b quark p^. All these distributions are plotted in Figs. 7-12 for the 
gu Zt signal at LHC Run L. 

We observe that the backgrounds are very concentrated at low pr^s. This makes convenient 
to use two different sets of cuts 1 and 2 for the 7^ and a^i, couplings, given in Table 5. These 
cuts, especially that on Ht, are very efficient for reducing the enormous background as 
can be observed in Table 6. For the a/^i, couplings in Run H, requiring very large transverse 
energy reduces the background by more than four orders of magnitude while retaining 
26% of the signal. As in the previous Subsection, the leptonic trigger has little effect in 
Run H. In fact this is smaller than the statistical fluctuation of the Monte Carlo. If no 
signal is observed, we find from Table 6 X^y^ < 0.036, < 0.076, < 0.015, Ktc < 0.045 
after Run L and Xtu < 0.020, Xt^ < 0.041, Ktu < 0.0076, Ktc < 0.025 after Run H. 

Variable Run L Run H 





Set 1 


Set 2 


Set 1 


Set 2 




70-90 


70-90 


70-90 


70-90 




160-190 


160-190 


160-190 


160-190 


Hx 


> 200 


> 450 


> 200 


> 600 


rmax 

Pt 


> 50 


> 50 


> 50 


> 50 


I, max 
Pi 


> 30 


> 30 


> 30 


> 30 


Pt 








> 30 



Table 5 

Kinematical cuts for the l'^l~jjb decay channel. The masses, energies and momenta are in GeV. 



3.3 bblvb signal 

This decay mode has a larger branching ratio than l~^l~li'b, but the need to tag two 
additional 6's and the trigger cuts reduce this advantage. On the other hand the modes 
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Run L 



Run H 





before 
cuts 


Set 1 
cuts 


Set 2 
cuts 


before 
cuts 


Set 1 
cuts 


Set 2 
cuts 


gu Zt{ji^) 


11.3 


9.9 




112 


98.6 




gc Zt (7^) 


2.6 


2.2 




25.8 


22.2 




gu Zt (a^^) 


26.5 




12.1 


265 




68.6 


gc Zt (o-^j,) 


4.6 




1.4 


46.3 




6.3 


Zjjj 


15600 


192 


6.6 


156000 


1920 


13.9 


Zbbj 


3660 


42.5 


1.5 


35900 


425 


3.3 


ZWj 


31.0 


3.7 


0.3 


309 


37 


0.6 



Table 6 

Number of l~^l~jjb events before and after the kinematical cuts in Table 5 for the Zt signal and 
backgrounds. We use Xtq = 0.02 and Ktq = 0.02. 

with few leptons have in general huge backgrounds at LHC, which can be reduced mainly 

by b tagging. If wc tag only one b quark we are considering the signal jjli'b, which has 
a branching ratio five times larger and gives nontrivial constraints at Tevatron. However, 
at LHC energies the process gb — > Wt — > jjli'b has a large cross section and can mimic 
the signal if the two non-6 jets have an invariant mass consistent with the Z mass. The 
tt WbWb —>■ jjblub cross section with one jet missed is even larger. Requiring three b 
tags both backgrounds become manageable. Other sizeable background is the small ZWj 
production. Wbbj production would be very large if we would only require two b jets. 
Requiring three b tags reduces this background to a moderate number of events due to 
the b misidentification factor of 0.01. Finally Wjjj becomes unimportant because it is 
suppressed by a factor of 10~^ accounting for the three b mistags. 

In this channel we have to identify the two h qiiarks resulting from Z bb. There are 
three pairs of tagged 6 jets, and we choose the one with invariant mass M^"^ closest to 
Mz- The other b is assigned to the top quark, and then the reconstructed top mass m^^'^ 
is calculated as for I'^l'lub. Other interesting variables are Ht and pf . 

A convenient set of cuts to improve the signal to background ratio is given in Table 7. In 
Table 8 we gather the number of bblub events for the signal and backgrounds before and 
after these kinematical cuts. We have used for the signal X^g = 0.02, n^g = 0.02. Note that 
although we have generate the ZWj background with the Z on-shell and hence its 
distribution is sharply peaked around Mz, this has little effect because this background 
is small. From Table 8 we obtain, if no signal is observed, Xm < 0.056, Xtc < 0.12, 
Ktu < 0.035, Ktc < 0.086 after Run L and Xtu < 0.035, Xtc < 0.083, Ktu < 0.019, 
Ktc < 0.050 after Run H. 
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Variable Run L Run H 
Mf"" 80-100 80-100 

mf= 160-190 160-190 
Ht > 240 > 300 
> 50 

Table 7 

Kinematical cuts for the hhlvh decay channel. The masses, energies and momentum are in GeV. 



Run L Run H 





before 
cuts 


after 
cuts 


before 
cuts 


after 
cuts 


gu Zt{j^) 


3.5 


2.4 


28.8 


13.3 


gc Zt (7^) 


0.8 


0.5 


6.3 


2.4 


gu Zt {a^y) 


8.0 


5.9 


71.4 


46.7 


gc Zt {a^t.) 


1.4 


1.0 


11.9 


6.7 


tt 


1790 


62.8 


14600 


345 


Wt 


21.0 


8.3 


161 


36.2 


ZWj 


11.1 


2.4 


93.1 


8.3 


Wbbj 


115 


1.5 


924 


4.3 



Table 8 

Number of bblub events before and after the kinematical cuts in Table 7 for the Zt signal and 
backgrounds. We use Xtq = 0.02 and Ktq = 0.02. 

3.4 i^i^jjh signal 



Let us conclude this Section with a short discussion of the vyjjh channel. At Tevatron 
this is the most interesting mode due to its relatively large branching ratio, 13.6%, its 
moderate background and the relatively low luminosity of the collider. However, its Wt 
and tt backgrounds grow very quickly with energy (see Table 9) and make this channel 
uninteresting at LHC. We will then focus for comparison on the Ztu couplings only. The 
results for Ztc are insignificant. 

We reconstruct the signal as in the l^l'jjb case, but with the charged leptons replaced 
by missing energy. We use for both Runs the kinematical cuts in Table 10. These are less 

restrictive than for the l^l~jjh mode because the Wt and ti backgrounds after missing 
the charged Icpton and the h quark mimic the signal and are irreducible. The Zjjj and 
Zbbj backgrounds are reduced by a factor of 80 (see Table 11). The bounds obtained are 
Xtu < 0.068, Ktu < 0.044 after Run L and Xtu < 0.042, < 0.021 after Run H. 
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Tevatron Run II 


LHC Run L 


Ratio 




0.148 


46 


1:310 




0.173 


112 


1:640 


Zjjj 


199 


23200 


1:120 


Zbbj 


74.1 


4590 


1:60 


wt 


3.5 


18500 


1:5300 


tt 


10.6 


54800 


1:5200 



Tabic 9 

Comparison between the number of fvjjb events without kinematical cuts for the Zt signal and 
backgrounds at Tevatron and LHC. We use Xm = 0.02 and Ktu = 0.02. 

Variable Runs L and H 
ml^ 160-190 
M^^ 70-90 
Ht > 180 

Table 10 

Kinematical cuts for the vujjh decay channel. The masses and energy are in GeV. 

Run L Run H 





before 
cuts 


after 
cuts 


before 
cuts 


after 
cuts 


gu Zt{ji^) 


46.0 


41.2 


201 


178 


gu Zt {a^t,) 


112 


101 


759 


681 


tt 


54800 


46200 


143500 


123400 


Wt 


18500 


16100 


41300 


37300 


Zjjj 


23200 


279 


76400 


657 


Zbbj 


4590 


67 


14800 


125 



Table 11 

Number of uvjjh events before and after the kinematical cuts in Table 10 for the Zt signal and 
backgrounds. We use Xtu = 0.02 and Ktu = 0.02. 

4 7^ production 

In contrast with the Zt case, ^yt production is not reduced by branching fractions because 
the photon is stable. Moreover, since the photon is massless, it tends to be produced with 

larger pr^s. This effect is further enhanced by the q" factor in the anomalous a^i, coupling. 
Thus the larger '-ft cross section, in particular for large momenta, allows for a better 
separation of the signal from the background, and then for more precise measurements 
than in the previous cases. 
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There are two channels depending on the W decay mode, 'ylub and 'yjjb. We analyze 
them in turn. These processes constrain not only the '-ftq anomalous couplings but also 
the strong anomalous couplings gtq. The jlub signal gives also in this case the most precise 
limits. We also discuss them in detail. 

4-1 ^Ivh signal 

The leptonic W decay gives a clean 7/Z/6 signal where the only background is ^Wj produc- 
tion with the jet misidentificd as a h. This case is then analogous to the l^l~lvh signal but 
with the pair replaced by the photon. The interesting kinematical variables are m^^^, 
defined as in the l'^l~lub channel, Ht and pj^. The corresponding distributions are plotted 
in Figs. 13-15. As emphasized above this signal can also be produced via gtq anomalous 
couplings. In this case the o"^,^ vertex couples the initial and not the final states in the 
s-channel, and the corresponding ml^'^, Ht and pj. distributions in the same Figures are 
different. Considering the minimum AR between the photon, the charged lepton and the 
jet, Ai?™™, is also useful to constrain these strong couplings. The signal for both pro- 
cesses and the background distributions are shown in Fig. 16. It is then convenient to 
apply different sets of cuts for the electromagnetic and strong couplings. In Table 12 we 
gather both sets, 1 (2) for '-ftq {gtq). The total number of events before and after these 
cuts are collected in Table 13, where we have used for the signals A^^ = 0.01, Qq = 0.01. 
If no signal is observed, we obtain Ai„ < 0.0048, < 0.013, Ctu < 0.0034, Qc < 0.0069 
after Run L and At„ < 0.0021, A^ < 0.0065, Qu < 0.0018, (tc < 0.0037 after Run H. 



Variable Run L Run H 





Set 1 


Set 2 


Set 1 


Set 2 


j^rec 


160-190 


160-190 


160-190 


160-190 


Ht 


> 300 


> 200 


> 300 


> 200 


7 
Pt 


> 100 




> 200 








> 0.6 




> 0.6 



Table 12 

Kinematical cuts for the jlub decay channel. The masses, energies and momenta are in GeV. 
4.2 jjjb signal 

The 7jj6 mode is analogous to the l'^l~jjb signal with the lepton pair replaced by the 
photon, and analogously to the Zt case this channel is less restrictive than the ^ylvb mode. 
Thus, although the hadronic W branching ratio is larger than the leptonic one, the 'yjjb 
backgrounds 'yjjj and '~fbbj are much larger than the ^lub ones. In order to reduce them 
we exploit the a^i, behaviour of the signal and systematically require large momenta, i. 
e., large Ht, Pt, Pt^ large photon energy E'^ and minimum jet transverse momentum 
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Run L 






Run H 








before 
cuts 


Set 1 
cuts 


Set 2 
cuts 


before 
cuts 


Set 1 
cuts 


Set 2 
cuts 


gu 


— > 7t (7*^) 


43.3 


29.3 




423 


179 




gc 




7.5 


4.2 




73.3 


19.5 




gu 


it (gtu) 


167 




118.5 


1470 




1190 


gc 


it igtc) 


43.2 




29.1 


368 




291 




iWqu 


140 


2.5 


16.8 


1250 


3.3 


168 




iWqa 


111 


2.8 


14.5 


980 


5.8 


145 




iWg 


56.7 


1.2 


6.3 


420 


2.4 


63 



Table 13 

Number of jlub events before and after the kinematical cuts in Table 12 for the Zt signal and 
backgrounds. We use Xtq = 0.01 and Qq = 0.01. 

p™''^, in addition to the usual requirements on M^y^ and m^*^'^ (see Figs. 17-23). The signal 
distributions have the same shape as for ■ylub, while the jjjj and •jbbj backgrounds are 
similar to Zjjj and Zbbj but tend to be more peaked at low prs. This is due to the 
masslessness of the photon. However, this effect is reduced by the trigger requirement of 
Pt ^ 40. A convenient set of kinematical cuts is given in Table 14, and the number of 
signal and background events before and after applying these cuts in Table 15. We also 
take into account the small iWj background for comparison. The effect of the kinematical 
cuts, especially at Run H, is impressive: the background is reduced by more than 10~^ 
while retaining 5 — 10% of the signal. This allows to obtain competitive bounds at least 
on 7tM couplings, Xtu < 0.0059, < 0.020 after Run L and < 0.0033, A^ < 0.012 
after Run H. 



Variable 


Run L 


Run H 


]\/rrec 


70-90 


70-90 


j^rec 


160-190 


160-190 


Ht 


> 540 


> 700 


7 
Pt 


> 230 


> 300 


E-r 


> 300 






> 30 


> 40 


P^T 


> 50 


> 60 



Table 14 

Kinematical cuts for the ^yjjb decay channel. The masses, energies and momenta are in GeV. 
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Run L Run H 





before 
cuts 


after 
cuts 


before 
cuts 


after 
cuts 


gu Zt {Gfj,^) 


102 


19.4 


970 


99.4 


gc Zt {ant,) 


18.5 


1.7 


170 


7.9 


ijjj 


466000 


6.9 


2070000 


15.3 


-fbbj 


92000 


2.0 


403000 


5.2 




659 


0.4 


3430 


1.0 



Table 15 

NTimbcr of ^jjh events before and after the kinematical cuts in Table 14 for the Zt signal and 
backgrounds. We use Aj^ = 0.01. 

5 Summciry 

We have studied Zt and ^t production via top FCN couplings at LHC. These processes 
manifest as 5 and 4 body final states, varying the statistical significance of the different 
channels (see Table 2) with the energy and luminosity of the collider. Eventually at LHC 
the best hmits on top FCN couphngs will be derived from Zt — > I'^l'lvh and ^t ^Ivb. 
In Table 16 we gather the corresponding values for integrated luminosities of 10 fb~^ (Run 
L) and 100 fb~^ (Run H). In order to compare the reach of the different decay modes, we 
collect in Table 17 the bounds on the Ztu anomalous couplings and expected in 
Run H for the most significant signals. At Tevatron with a luminosity of 109 pb~^ (Run 
I) and 2 fb~^ (Run II) the most significant channels are Zt — > vvjjh and ^t — > ^Ifh [6]. 
In both Runs the most sensitive decay mode with the Z boson decaying hadronically is 
jjlvh. This variation of the relevance of the signals is mainly due to the small statistics 
available at Tevatron and to the background increase at LHC, in part consequence of the 
larger h content of the proton. The statistics penalizes the channels with few events when 
the backgrounds are negligible, whereas the large backgrounds make uninteresting the less 
significant signals. 

A few concluding remarks: 

• If no signal is observed, Zt and ^t production at the LHC will allow to obtain indepen- 
dent bounds on the anomalous top couplings to up and charm quarks. 

• Limits derived from top decays arc less precise but comparable, especially taking into 
account that they have been obtained using a conservative estimate. 

• The bounds on the anomalous cr^,^ couplings in Table 16 seem more stringent than those 
on 7^ couplings because they are normalized to rut which we take equal to 175 GeV in 
Eq. (1), whereas the energies probed are significantly larger. If instead we would have 
used A = 1 TeV, the constraints on a^^ couplings would have looked less stringent. 

• Strong anomalous couplings gtq are more precisely constrained by single t production 
[16]. 
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Signal Run Xtu Xtc Ktu i^tc Mc Ctu Ctc 

l+l'lvh RunL 0.022 0.045 0.014 0.034 — — 0.0069 0.017 

RunH 0.011 0.023 0.0063 0.016 — — 0.0030 0.0078 

7^1/6 RunL _ _ _ _ 0.0048 0.013 0.0034 0.0069 

RunH _ _ _ _ 0.0021 0.0065 0.0018 0.0037 

Table 16 

Most stringent bounds on the anomalous top couplings in Eqs. (1), (2) from single top production 
in association with a Z boson or a photon at the LHC. 



Channel 








0.011 


0.0063 


m-jjb 


0.020 


0.0076 


bblub 


0.035 


0.019 


uvjjb 


0.042 


0.021 



Tabic 17 

Limits on the Ztu anomalous couplings Xtu and Ktu for the most significant decay channels at 
LHC Run H. 



• All these bounds may be too optimistic at the end and a real simulation of the exper- 
imental conditions without neglecting a priori other possible small backgrounds and 
the uncertainties associated with the structure functions is necessary to obtain better 

estimates. 

• The Vt signal is undcrstimated because ti production with a FCN top decay into Wh 
and an undetected light quark is not included. 

• Multilepton signals are expected from other processes, for example they are character- 
istic in gauge mediated supersymmetry breaking models [33]. However if their origin is 
Zt and 7t production, the fixed ratios between the different Z and t decay modes will 
allow to establish their origin. 

• The best hmits on top FCN couphngs are expected to be obtained at LHC because it will 
produce many more tops than other planned machines. However, the clean environment 
of e^e~ colliders makes them complementary, particularly if new physics is observed. 

• In any case, h tagging plays an essential role in tracing tops and reducing backgrounds. 

• The large number of tops to be produced at future colliders, the present lack of precise 
knowledge of the top properties and the widely spreaded idea that new physics must 
first manifest in the heaviest family make top physics particularly important. 

• In a general effective Lagrangian approach with only the SM light degrees of freedom the 
lowest order top FCN couplings are all dimension 6 [8,10]. After electroweak symmetry 
breaking they generate the dimension 4 and 5 couplings we have considered, which are 
the lowest dimension top FCN vertices with only one gauge boson. Thus, testing these 
couphngs one expects to probe a large class of SM extensions. 
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Fig. 3. Reconstructed Z mass M^'^ distribution for the l^l jjb signal at LHC with Gaussian 
energy smearing and detector and trigger cuts. 
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Fig. 4. Reconstructed top mass ml^'^ distribution before kinematical cuts for the gu — > l^l Ivb 
signal and background in LHC Run L. We use Xtu = 0.02, Ktu = 0.02 
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Fig. 5. Total transverse energy Hx distribution before kinematical cuts for the gu 
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Fig. 10. pj.™*^^ distribution before kinematical cuts for the gu — > jjb signal and backgrounds 
in LHC Run L. We use Xtu = 0.02, Ktu = 0.02 
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Fig. 17. Reconstructed W mass distribution before kinematical cuts for the gu 

signal and backgrounds in LHC Run L. We use Xtu = 0.01. 
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Fig. 18. Reconstructed top mass rriY'^ distribution before kinematical cuts for the gu — ^jjb 
signal and backgrounds in LHC Run L. We use Xtu = 0.01. 
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Fig. 19. Total transverse energy Ht distribution before kinematical cuts for the gu 
signal and backgrounds in LHC Run L. We use Xtu = 0.01. 
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Fig. 20. distribution before kinematical cuts for the gu — ^ ^jjh signal and backgrounds in 
LHC Run L. We use Xtu = 0.01. 
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Fig. 21. E'^ distribution before kinematical cuts for the gu 
LHC Run L. We use Xtu = 0.01. 
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Fig. 22. pjJ™ distribution before kinematical cuts for the gu 
LHC Run L. We use Xtu = 0.01. 
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Fig. 23. distribution before kinematical cuts for the gu jjjb signal and backgrounds in 
LHC Run L. We use Xtu = 0.01. 
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